Method and apparatus for adaptively compressing a video signal in accordance with the valves of individual pixels

ABSTRACT

On a transmitting side, a digital video signal comprising respective pixel samples (each of which may include eight bits) is received, and a set of elementary pixel samples determined from the received pixel samples at a predetermined rate are transmitted. The values of respective subject pixel samples other than the elementary ones are predicted, and respective predictive errors of the predicted values are detected from present values of the received pixel samples. First flags are then generated individually for each subject pixel sample whose predictive error is greater than a threshold value, and second flags are generated individually for each subject pixel sample whose predictive error is less than or equal to the threshold value. The present values of the subject pixel samples having the first flags are transmitted, and compressed data (including the second flags) is transmitted for the subject pixel samples having the second flags. On a receiving side, values of the subject pixel samples represented by the compressed data with the second flags are predicted, and the subject pixels are interpolated with pixel samples having a present value equal to the predicted value. A decoded video signal, including the transmitted elementary pixel samples, the respective subject pixel samples having the first flags, and the respective interpolated subject pixel samples, is then outputted.

FIELD OF THE INVENTION:

The present invention relates to a video signal compressiveencoding/decoding method and an apparatus for carrying out such method,and more particularly to an improved method and apparatus for encodingand decoding a video signal compressed adaptively in accordance with thevalues of individual pixels.

BACKGROUND OF THE INVENTION

In transmission of a video signal, it is conventional to compress theamount of transmission data in comparison with the amount of originaldata. An exemplary conventional method for such performing suchcompression is a subsampling technique which reduces a samplingfrequency by thinning out the individual pixel samples of a digitizedoriginal video signal at predetermined intervals. More specifically, asdisclosed in Japanese Patent Laid-open No. 57 (1982)-78290, theindividual pixel samples of the video signal are so thinned out that thesampling rate is reduced by one-half, and the data of the non-thinnedpixel samples are transmitted while a flag indicating the position ofthe non-thinned sample to be used for interpolation on the receivingside is also transmitted with regard to the thinned sample.

One problem with such a conventional subsampling process in which thesubsampling pattern is not changed, is that, in the contour or the likeof a subject image including high frequency components, the reproducedimage exhibits conspicuous quality deterioration. Particularly when thesubsampling rate is lowered, such image quality deterioration isextremely great.

For the purpose of solving the problems mentioned, the present applicantpreviously proposed an improved method which divides the image of oneframe into a plurality of segmental blocks then selects a suitable oneout of a plurality of prepared sampling patterns, and transmits,together with the pixel sample to be transmitted, an identification coderepresenting the selected sampling pattern.

However, in this method, the number of kinds of prepared samplingpatterns must be limited for suppressing the redundancy derived from theidentification code, and consequently the method is not suitable forprocessing every image. Moreover this method has the disadvantage thatthe segmentation operation may bring about block distortion.

OBJECTS AND SUMMARY OF THE INVENTION

It is therefore an object of the present invention to provide a videosignal compressive encoding/decoding method which is capable of solvingthe known problems mentioned, and also an apparatus for carrying outsuch method.

In particular, a principal object of the present invention resides inproviding an improved video signal compressive encoding/decoding methodand apparatus which minimizes the quality deterioration of thereproduced image.

Another object of the invention is to provide a video signal compressiveencoding/decoding method and apparatus adapted to diminish the qualitydeterioration of an image reproduced on a block basis.

A further object of the invention is to provide a video signalcompressive encoding/decoding method and apparatus capable ofaccomplishing a real-time process adequate for a motion image.

A further object of the invention resides in providing a video signalcompressive encoding/decoding method and apparatus which can maintainsubstantially fixed the unitary amount of generated information to beprocessed.

Still another object of the invention is to provide a video signalcompressive encoding/decoding method and apparatus requiring nolarge-capacity buffer memory.

Yet another object of the invention is to provide a video signalcompressive encoding/decoding method and apparatus adapted to reduce theamount of generated information that may otherwise be increased by somenoise component.

According to one aspect of the inventive video signal compressiveencoding method and apparatus, a digital video signal represented byrespective pixel samples is received, a set of elementary pixel sampleshaving a predetermined period is determined from the pixel samples,values corresponding to those pixel samples other than the elementarypixel samples (referred to herein as "non-elementar" pixel samples, or"subject" pixel samples) are predicted, a predictive error for eachpredicted value is determined from the present values of the pixelsamples, the respective predictive errors are compared with a thresholdvalue, respective flags are generated when the respective predictiveerrors do not exceed the threshold value, and an encoded video signalincluding the following elements is transmitted: the elementary pixelsamples, the present value of each non-elementary pixel sample whosepredictive error is greater than the threshold value, and a compresseddata (including at least one of the flags) signal for eachnon-elementary pixel sample whose predictive error is less than or equalto the threshold value.

Another aspect of the invention is a method and an apparatus fordecoding the encoded video signal described above, which are adapted forreceiving the transmitted elementary pixel samples, the transmittednon-elementary pixel sample present values, and the transmittedcompressed data signals; determining an interpolated pixel sample valuefor each compressed data signal by an interpolation operation; andoutputting a decoded video signal including the transmitted elementarypixel samples, the transmitted non-elementary pixel sample presentvalues, and the interpolated pixel sample values.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 schematically illustrates a video signal that may be processed inaccord with the compressive encoding method of the present invention;

FIG. 2 is a schematic block diagram of an exemplary video signalcompressive encoding apparatus embodying the present invention;

FIG. 3 is a block diagram of an exemplary subsampling encoder employedin the video signal compressive encoding apparatus of the invention;

FIG. 4 is a block diagram of an exemplary threshold determinationcircuit employed in the subsampling encoder of the video signalcompressive encoding apparatus of the invention;

FIG. 5 is a block diagram of an exemplary decoder for a video signalcompressively encoded by the subsampling encoder shown in FIG. 3;

FIG. 6 is a block diagram of an exemplary interpolation circuit employedin the sampling decoder of FIG. 5;

FIG. 7 is a block diagram of an exemplary nonlinear filter shown in FIG.2;

FIG. 8 is a graph schematically illustrating the operation of thenonlinear filter shown in FIG. 7;

FIG. 9 is a block diagram of an exemplary subsampling encoder employedin another embodiment of the video signal compressive encoding apparatusof the invention;

FIG. 10 is a block diagram of an exemplary subsampling decoder employedin another embodiment of the video signal compressive decoding apparatusof the invention;

FIG. 11 is a flow chart of the video signal compressive encodingalgorithm executed in the subsampling encoder of FIG. 9; and

FIG. 12 is a flow chart of the decoding algorithm executed in thesubsampling decoder of FIG. 10.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

Hereinafter a video signal compressive encoding/decoding method of thepresent invention and an apparatus contrived for carrying out suchmethod will be described in detail with reference to the accompanyingdrawings. First, the inventive video signal compressive encoding methodwill be described.

In FIG. 1, individual pixel samples existing in partial regions of onefield of an input digital video signal are represented by symbols ⊚ o,□, Δ and x respectively, wherein the horizontal spacing between thepixel samples corresponds to the sampling period, and the verticalspacing corresponds to the line interval. Denoted by a sign o is anelementary pixel sample occurring in one out of every four lines, and inone out of every four pixels in each line in which it occurs. Theelementary pixel samples thus included at a rate of 1 per 16 pixelsamples are entirely transmitted without being thinned out.

Meanwhile the other ("non-elementary") pixel samples represented bysigns o, □, Δ and x other than the elementary ones are adaptivelythinned out. That is, in the following sequence, the value of eachnonelementary pixel sample is compared with the predicted value thereofbased on the value of its peripheral reference pixel sample, and whenthe predicted residue ε is smaller than a threshold value TH, thesubject pixel sample is not transmitted ("thinned out") and instead anidentification code indicating the thin-out of such subject pixel sampleis transmitted. On the other hand, in case the predicted residue ε is inexcess of the threshold value TH, the subject pixel sample istransmitted without being thinned out.

The threshold value TH is determined by taking into consideration theimage quality deterioration that may be induced in the reproduced imageduring the decoding operation (to be described below) when the value ofthe thinned-out pixel sample is interpolated using one or moreperipheral reference pixel samples associated with the thinned-out pixelsample.

For example, the value of the non-elementary pixel sample a3 denoted byo (in FIG. 1) is compared with the average value 1/2 (a1+a5) of twoperipheral elementary pixel samples a1 and a5 at positions spaced apartby 2 lines upward and downward respectively in the same field.Similarly, the values of non-elementary pixel sample e3 is compared withthe average value of two peripheral elementary pixel samples e1 and e5.

The value of the pixel sample c1 denoted by □ is compared with theaverage value 1/2 (a1+e1) of two peripheral pixel samples a1 and e1 atpositions spaced apart by two pixel samples leftward and rightward onthe same line. Similarly, the values of subject pixel samples c3 and c5are compared with the average values of peripheral pixel samples a3 ande3, and a5 and e5, respectively.

The value of a pixel sample 2 denoted by Δ is compared with the averagevalue 1/2 (a1+a3) of two peripheral pixel samples a1 and a3 at positionsspaced apart by one line upward and downward respectively in the samefield. Similarly, the values of subject pixel samples c2, e2, a4, c4 ande4 are compared with the average values of peripheral pixel samples c1and c3, e1 and e3, a3 and a5, c3 and c5, and e3 and e5, respectively.

The value of the pixel sample b1 denoted by x is compared with theaverage value 1/2 (a1+c1) of two peripheral pixel samples a1 and c1 atpositions spaced apart by one pixel sample leftward and rightwardrespectively on the same line. Similarly, the values of subject pixelsamples b2, b3, b4, b5, d1, d2, d3, d4 and d5 are compared with theaverage values of peripheral pixel samples a2 and c2, a3 and c3, a4 andc4, a5 and c5, c1 and e1, c2 and e2, c3 and e3, c4 and e4, and c5 ande5, respectively.

When the pixel samples are adaptively thinned out as mentioned above,the amount of output data depends upon the content of the image.Accordingly, if the video signal thus processed by compressive encodingis recorded and reproduced by means of a tape recorder or the like,there occurs a variation in the amount of data per track. On the otherhand, if the data rate is maintained constant in a recording mode, theunitary length is changed as a result to consequently emplicate theediting and so forth.

For the purpose of eliminating the disadvantage described in thepreceding paragraph, buffering is carried out to maintain constant theamount of data recorded per track. In such embodiment of the compressiveencoding method, the number of pixel samples to be thinned out ischanged by varying the threshold value TH of subsampling encoder 1(shown in FIG. 2) in accordance with the content of the video signal perfield.

That is, the number of pixel samples thinned out becomes greater if thethreshold value TH is increased, consequently reducing the amount oftransmitted information. On the contrary, the number of pixel samplesthinned out becomes smaller if the threshold value TH is decreased,consequently increasing the amount of transmitted information. Thereforebuffering is accomplished by controlling the threshold value TH.

As shown in FIG. 2, the video signal compressive encoding apparatus ofthe present invention consists fundamentally of a subsampling encoder 1.So that the threshold value TH received by subsampling encoder 1 isadaptively changed in accordance with the content of the input image,there is provided a threshold determination circuit 2. To preventfailure to attain a desired compression rate due to some noise componentincluded in the input video signal, nonlinear filter 3 is provided forremoving such noise component Accordingly, the compressive encodingapparatus comprises, as shown in FIG. 1: an input terminal 4 suppliedwith a digital video signal where each pixel sample is quantized with 8bits at a sampling frequency of 13.5 MHz; a nonlinear filter 3 connectedto the input terminal 4; a subsampling encoder 1 supplied with thefiltered digital video signal from which the noise has already beenremoved in nonlinear filter 3; an output terminal 5 at which thecompressed video signal encoded by the subsampling encoder 1 isobtained; a predictive error filter 6 constituted partially in the samemanner as the subsampling encoder 1 and supplied with the noise-removeddigital video signal from the nonlinear filter 3; a thresholddetermination circuit 2 for deciding the threshold value TH of thesubsampling encoder 1 adaptively in accordance with the predictive errorε outputted from the predictive error filter 6; and a delay circuit 7inserted between the nonlinear filter 3 and the sub-sampling encoder 1.Circuit 7 includes a field memory and so forth to compensate for theprocessing times of predictive error filter 6 and thresholddetermination circuit 2.

In case the threshold value TH of the subsampling encoder 1 need not bechanged adaptively, the predictive error filter 6, thresholddetermination circuit 2, and delay circuit 7 may be omitted from theFIG. 2 apparatus.

Next, the specific constitution of the individual circuit blocks in theFIG. 2 embodiment of the video signal compressive encoding apparatuswill be described in sequence. First, subsampling encoder 1 will bedescribed with reference to FIG. 3.

In FIG. 3, a digital video signal is fed via the delay circuit 7 to aninput terminal 101 of the subsampling encoder 1. The terminal 101 isconnected to line delay ("LD") circuits 102, 103, 104 and 105 in acascade connection. Sample delay ("SD") circuits 106 and 107 are alsoconnected in series to input terminal 101. Sample delay circuits 108 and109 are connected in series to the output side of line delay circuit102, and sample delay circuits 110, 111, 112 and 113 are connected inseries to the output side of line delay circuit 103. Sample delaycircuits 114 and 115 are connected in series to the output side of linedelay circuit 104, and sample delay circuits 116 and 117 are connectedin series to the output side of line delay circuit 105. Each of suchline delay circuits 102, 103, 104 and 105 has a delay time correspondingto one horizontal scanning interval, and each of the sampling delaycircuits 106 through 117 has a delay time corresponding to one samplinginterval. The sample data of a plurality of pixels included in apredetermined two-dimensional area of the television image can beextracted simultaneously by a combination of the line delay circuits 102through 105 and the sample delay circuits 106 through 117.

The output side of tee sample delay circuit 111 corresponds to thesubject pixel sample out of those extracted simultaneously.

Each of selectors 118 and 119 has five input terminals and, in responseto a selection signal received from a terminal 120 synchronously with asampling clock pulse, selectively provides at its output terminal one ofthe input data fed to such five input terminals respectively.

Although no detailed explanation is necessary with regard to this stage,it will be appreciated that the selection signal should be generated soas to ensure execution of the below-mentioned selection operation inaccordance with an adequate timing generator circuit.

The first input terminal of selector 118 is supplied with the outputdata of sample delay circuit 107, and the firs input terminal ofselector 119 is supplied with the output data of sample delay circuit117. Accordingly, in case the subject pixel sample is any one denoted byo, the input data supplied to the respective first input terminals ofthe selectors 118 and 119 are selected as peripheral reference pixelsamples. The second input terminals of the selectors 118 and 119 aresupplied, respectively, with the output data of the sample delaycircuits 109 and 115. Accordingly, in case the subject pixel sample isany one denoted by Δ, the input data supplied to the respective secondinput terminals of the selectors 118 and 119 are selected as peripheralreference pixel samples. The third input terminals of the selectors 118and 119 are supplied with the output data of line delay circuit 103 andsample delay circuit 113. Accordingly, in case the subject pixel sampleis any one denoted by □, the input data supplied to the respective thirdinput terminals of the selectors 118 and 119 are selected as referencepixel samples. The fourth input terminals of the selectors 118 and 119are supplied with the output data of sample delay circuits 110 and 112.Accordingly, in case the subject pixel sample is any one denoted by x,the input data supplied to the respective fourth input terminals of theselectors 118 and 119 are selected as peripheral reference pixelsamples. The fifth input terminals of the selectors 118 and 119 aresupplied with the output data of sample delay circuit 111. In case thesubject pixel sample is any one denoted ⊚ , both the selectors 118 and119 select the elementary pixel sample directly.

The output data of the selectors 118 and 119 are supplied to averagingcircuit 121, which generates a signal representing the average data ofthe two peripheral reference pixel samples selected individually by theselectors 118 and 119. Such average data, and the data of the subjectpixel sample emerging from sample delay circuit 111, are both suppliedto subtraction circuit 122. The residual data obtained from circuit 122is supplied to absolute value circuit 123 so as to be converted into anabsolute value. Subsequently the output data from the absolute valuecircuit 123 is supplied to comparator 124 in which it is compared withthe threshold value obtained from the terminal 125.

The output data of the absolute value circuit 123 represents thepredictive error ε generated when the value of the subject pixel sampleis predicted from the average of the values of the two peripheral pixelsamples as mentioned previously. If the predictive error ε is less thanthe threshold value TH, this signifies that the subject pixel sample maybe thinned out. In this case, appropriate control data (which maycomprise a single bit) indicating this status emerges from comparator124 (i.e., the bit emerging from comparator 124 is a binary "1"). If thepredictive error ε exceeds the threshold value TH, this signifies thatadequate interpolation is impossible on the receiving side, so that thcontrol data from the comparator 124 is turned to binary "0". Thecontrol data thus obtained serves to execute on/off control of gatecircuit 126 which is supplied with the output data of the sample delaycircuit 111. When the control data from circuit 124 is "0", the gatecircuit 126 is turned on so that the original data of the subject pixelsample is supplied to output terminal 127. However, when the controldata from circuit 124 is "1", the gate circuit 126 is turned off so thatthe original data of the subject pixel sample is not supplied to outputterminal 127. The control data is also supplied to output terminal 128,from which terminal it may be transferred together with the subsampleddata of the pixel sample. That is, the output terminals 127 and 128 ofthe subsampling encoder 1 may be connected to a framing circuit (notshown in FIG. 3), where the pixel example data and the control data arecombined with each other. This combined signal (consisting of nine bitsper pixel) is transmitted to output terminal 5 of FIG. 2 when the pixelsample is not thinned out; but when the pixel sample is thinned out,merely the control data (one bit per pixel) is transmitted to outputterminal 5.

As described above, the subsampling is performed in accordance withwhether the predictive error ε is greater than the threshold value ornot in regard to each subject pixel sample. In other words, datatransmission or thinning-out is controlled not on the basis of block butadaptively on the basis of each pixel sample, which is the minimum unit.In making a decision as to whether the thinning-out operation isperformed or not in conformity with the predictive error ε, the actualdata is used in place of the interpolation data to consequently enablereal-time processing without undesired repetition.

Predictive error filter 6 may be substantially the same as subsamplingencoder 1 shown in FIG. 3. However, since filter 6's function is merelyto generate the predictive error ε (corresponding to the output ofabsolute value circuit 123 in FIG. 3), filter 6 will differ from encoder1 in that comparator 124 and gate circuit 126 will be omitted fromfilter 6.

Next, the constitution of threshold determination circuit 2 will bedescribed with reference to FIG. 4.

In FIG. 4, an input terminal 201 is supplied with the predictive error εobtained from the predictive error filter 6. The predictive error ε,when composed of 8 bits, may take any value in the range from 0 through255.

The predictive error ε is fed as an address signal to a frequencydistribution memory 203 via a selector 202. Also a sampling clock froman input terminal 204 is fed as a write/read signal to the frequencydistribution memory 203 via a selector 205. In response to thewrite/read signal the frequency distribution memory 203 performs itsoperation in a read-modified-write mode to write the data immediatelyafter reading out with respect to the same address.

Frequency distribution memory 203 selectively receives, at its inputterminal via a selector 207, either "0" or the value obtained byincrementing the output of the memory 203 by one in an adder 206.

To commence operation, frequency distribution memory 203 is initializedso that it's entire contents are set to zero. Then when the predictiveerror ε is fed to memory 203 as an address, the address data (zero inthe initial state) is read out and fed to the adder 206, where the valueis incremented and then is rewritten in the same address. Thus, when thepredictive errors ε are fed during one field, the pixel samplefrequencies for producing the individual predictive errors ε are storedin the addresses 0 to 255 of frequency distribution memory 203.

The threshold value TH is determined by using the frequency distributiontable stored in the memory 203. The operation of determining thethreshold value is executed within, for example, the vertical blankinginterval. During such threshold determination, the contents of counter208 is sequentially incremented from 0 to 255 in response to the clocksignal from input terminal 209 and is then fed as an address tofrequency distribution memory 203 via selector 202.

The clock signal from terminal 209 is fed as a read signal to memory 203via selector 205. The numbers of pixel samples represented by theindividual frequencies of the predictive errors ε stored in thefrequency distribution memory 203 are read out and fed to an accumulator210. Simultaneously the selector 207 selects zero data, which is thenwritten in the frequency distribution memory 203 to initialize it forprocessing the next field.

Accumulator 210 sequentially accumulates the frequencies of thepredictive errors ε from 0 toward 255. The output value of accumulator210 is fed to comparator 211, which is supplied with the required numberof thin-out pixel samples corresponding to the target rate, so that theoutput value of the accumulator 210 is compared with such requirednumber. When the output value of accumulator 210 has exceeded therequired number of thin-nut pixel samples, a latch pulse is generatedfrom comparator 211.

The output signal of counter 208 incremented from 0 toward 255 is fed tolatch circuit 212 and is thereby latched by the latch pulse generatedfrom comparator 211. Consequently, the value latched in latch circuit212 corresponds to the minimum of the predictive errors ε conformingwith the required number of thin-out pixel samples. The value thusobtained is taken out from the output terminal 213 as the thresholdvalue TH of the subsampling encoder 1.

Since the frequency of the predictive error 0 includes elementary pixelsas well as non-elementary pixels, the required number of thin-out pixelsamples is determined in consideration of such frequency.

Prior to giving a description of the nonlinear filter 3, the apparatusfor decoding the compressed video signal will be explained withreference to FIGS. 5 and 6.

FIG. 5 shows the circuit configuration of a subsampling decoder providedon the inventive system's receiving side (the reproducing side in arecording/reproducing apparatus). In FIG. 5, an input terminal 401 issupplied with the compressive-encoded digital video signal, while aninput terminal 402 is supplied with a sampling clock signal synchronizedwith the received data.

The input terminal 401 is connected in series to line delay circuits403, 404, 405 and 406. Serial-to-parallel converters 407, 408, 409, 410and 411 are connected respectively to the input terminal 401 and theoutputs of the line delay circuits 403, 404, 405, and 406. The datareceived from terminal 401 and the delayed data output from circuits 403through 406 are sequentially fed into serial-to-parallel converters 407through 411 in synchronism with the sampling clock signal, and he dataof four pixel samples are latched by the output signal of 1/4 frequencydivider 412. Upon input of the data of the next pixel sample, each ofcircuits 407 through 411 outputs the data of five pixel samples inparallel. Accordingly, at one timing instant, the twenty-five pixelsamples with reference numerals a1 through e5 shown in FIG. 1 areoutputted respectively from the serial-to-parallel converters 407through 411. For instance, the data of the four pixel samples a1, b1, c1and d1 from the line delay circuit 406 re latched in theserial-to-parallel converter 411, and the data of a total of five pixelsamples, including the next pixel sample e1, emerge simultaneously fromserial-to-parallel converter 411.

Of all twenty-five of the signals output from the serial-to-parallelconverters 407 through 411, samples a5, b5, c5, d5, and e5, and e1, e2,e3 and e4 include peripheral reference pixel data used forinterpolation, and the remaining sixteen pixels (the signals emergingfrom circuits 407 through 411 other than such peripheral referencepixels) are subjects to be interpolated. The interpolation circuits 413through 431 are structurally identical. FIG. 6 specifically shows theconstitution of the interpolation circuit 413 as an example.

Interpolation circuit 413 has input terminals 413a, 413b, 413c and anoutput terminal 413d. Input terminal 413a is supplied with the data(including one-bit control data) for one subject pixel, e.g. c5, to beinterpolated, while the input terminals 413b and 413c are supplied withthe data of the peripheral reference pixel samples e5 and a5 requiredfor interpolation. The pixel data from the input terminals 413b and 413care fed to the averaging circuit 413e, which then produces an outputsignal for the average value interpolation. The pixel sample data frominput terminal 413a and the output signal of averaging circuit 413e arefed to a selector 413f.

Selector 413f is controlled by the one-bit control data included in thepixel data from input terminal 413a and, when the control data is "1"representative of thinning out, selector 413f selects the output ofaveraging circuit 413e. When the control data is "0" representative oftransmission, selector 413f selects the pixel sample data from the inputterminal 413a. The output signal of selector 413f is obtained at theoutput terminal 413d.

In case the subject pixel samples are to be thinned out, theinterpolation values obtained respectively from the interpolationcircuits 413 through 431 are as follows:

Interpolation circuit 413: c5→1/2 (a5+e5)

Interpolation circuit 414: e4→1/2 (e3+e5)

Interpolation circuit 415: c4→1/2 (c3+c5)

Interpolation circuit 416: a4→1/2 (a3+a5)

Interpolation circuit 417: d4→1/2 (c4+e4)

Interpolation circuit 418: b4→1/2 (a4+c4)

Interpolation circuit 419: e3→1/2 (e1+e5)

Interpolation circuit 420: a3→1/2 (a1+a5)

Interpolation circuit 421: c3→1/2 (a3+e3)

Interpolation circuit 422: d3→1/2 (c3+e3)

Interpolation circuit 423: b3→1/2 (a3+c3)

Interpolation circuit 424: e2→1/2 (e1+e3)

Interpolation circuit 425: c2→1/2 (c1+c3)

Interpolation ciccuit 426: a2→1/2 (a1+a3)

Interpolation circuit 427: d2→1/2 (c2+e2)

Interpolation circuit 428: b2→1/2 (a2+c2)

Interpolation circuit 429: c1→1/2 (a1+e1)

lnterpolation circuit 430: d1→1/2 (c1+e1)

lnterpolation circuit 431: b1→1/2 (a1+c1).

With reference again to FIG. 5, the data of sixteen pixels included inthe output signals from the interpolation circuits 413 through 431, arefed respectively to parallel-to-serial converters 432, 433, 434 and 435at a rate of four pixels on the same line. In such parallel-to-serialconverters 432 through 435, the four post-interpolation pixel data arelatched respectively by the output signal from the 1/4 frequency divider412. Serial reproduced data are outputted from the parallel-to-serialconverters 432 through 435 synchronously with the sampling clock signalfed from terminal 402. It is a matter of course that the pixel datashown n FIG. 5 become different at the instant the next clock signal isgenerated from the 1/4 frequency divider 412. That is, the individualpixel data a1, a2, a3, a4 and a5 from the serial-to-parallel converters407 through 411 are replaced with pixel data e1, e2, e3, e4 and e5respectively.

The reproduced data from the parallel-to-serial converter 432 are fed toline delay circuit 436, whose output data are then fed to selector 437together with the reproduced data obtained from the parallel-to-serialconverter 433. Subsequently the output data of selector 437 are fed toline delay circuit 438, whose output data are fed to selector 439together with the reproduced data from parallel-to-serial converter 434.Thereafter the output data of selector 439 are fed to line delay circuit440, whose output data are then fed to selector 441 together with thereproduced data from the parallel-to-serial converter 435. Such linedelay circuits 436, 438, 440 and the selectors 437, 439, 441 areprovided for converting the sequence of the reproduced data to the samesequence executed in the television scanning, whereby the reproduceddata in the television scanning sequence are obtained at output terminal442 of selector 441.

Next, the constitution of nonlinear filter 3 will be described belowwith reference to FIG. 7. Nonlinear filter 3 serves for removal ofnoise, and principally comprises sample delay circuits 301 and 302,selectors 303 and 304, additive data generators 305 and 306, an adder307, a subtracter 308, comparators 309 and 310, and a discriminator 311.When a wave having polar values is formed in the one-dimensionalscanning line (horizontal) direction by three consecutive sampling dataas illustrated in FIG. 8, the levels of the individual sampling data arecompared with one another for selection, and one proper data isselectively outputted as replacement data.

The received digital video signal is fed to input terminal 312, which isconnected in series to sample delay circuits 301 and 302. Each of suchsample delay circuits 301 and 302 has a delay time equivalent to onesampling interval.

Suppose now that, as illustrated in FIG. 8, one pixel Pn in the digitalvideo signal is considered the subject pixel, and two pixels existing onthe anterior and posterior peripheries of such subject pixel Pn areregarded here as peripheral reference pixels Pn-1 and Pn+1.

The subject pixel Pn and the peripheral reference pixels Pn-1, Pn-1 areextracted by the aforementioned sample delay circuits 301 and 302. Atthe timing to feed the peripheral reference pixel Pn+1 to the inputterminal 312, the subject pixel Pn is obtained from sample delay circuit301, and the peripheral reference pixel Pn-1 from sample delay circuit302, respectively.

Selector 303 is supplied with both the input signal from input terminal312 and the output signal of sample delay circuit 302. Selector 303discriminates between the levels of the two input terminals, i.e. thedata of the two peripheral reference pixels Pn-1 and Pn+1, and outputsthe higher-level signal as a maximal value MAX to adder 307, whileoutputting the lower-level signal as a minimal value MIN to subtracter308.

Additive data generator 305 is connected to adder 307 inserted betweenselector 303 and comparator 309, while another additive data generator306 is connected to subtracter 308 inserted between selector 303 andcomparator 310. Additive data generators 305 and 306 are provided forrespectively changing the maximal value MAX and the minimal value MIN,which are outputted from the selector 303, in accordance with an offsetsignal fed from input terminal 314. An offset A1 generated from additivedata generator 305 is added to the maximal value MAX by adder 307 toproduce a maximal value MAXo (=MAX+Δ1). Meanwhile an offset Δ2 generatedfrom additive data generator 306 is subtracted from the minimal valueMIN by subtracter 308 to produce a minimal value MINo (=MIN-Δ2). Suchtwo offsets Δ1 and Δ2 are constants changed in proportion to thesampling density.

The output signal (i.e. maximal value MAXo) of adder 307 is fed to oneinput terminal of comparator 309 and also to the first input terminal ofselector 304. The output signal (i.e. minimal value MINo) of subtracter308 is fed to one input terminal of comparator 310 and also to thesecond input terminal of selector 304.

Meanwhile, the output signal (i.e. data Dpn of subject pixel Pn) ofsample delay circuit 301 is fed to the other input terminals ffcomparators 309, 310 and also to the third input terminal of selector304.

In comparator 309, the pixel data selected as the maximal value MAXo outof the peripheral reference pixels Pn-1 and Pn+1 is compared with thesubject pixel data Dpn outputted as a comparative value from sampledelay circuit 301, and the numerical relation between the two compareddata is fed as a comparison signal Sc₁ to discriminator 311.

In comparator 310, the pixel data selected as the minimal value MINo outof the peripheral reference pixels Pn-₁ and Pn+₁ is compared with thesubject pixel data Dpn outputted as a comparative value from sampledelay circuit 301, and the numerical relation between the two compareddata is fed as a comparison signal Sc₂ to discriminator 311. Since themaximal value MAXo and the minimal value MINo include the offsets Δ1 andΔ2 added thereto respectively, it follows that the subject pixel dataDpn is weighted in comparators 309 and 310.

On the basis of such comparison signals Sc₁ and Sc₂, discriminator 311decides the overall numerical relation among the maximal value MAXo, theminimal value MINo and the subject pixel data Dpn, and produces atwo-bit decision signal SJ for selecting the value of the intermediatelevel (hereinafter referred to as intermediate value).

In selection of such intermediate value, it is necessary to take intoconsideration the presence or absence of any noise that may besuperimposed on the subject pixel data Dpn.

For instance, when the subject pixel Pn is free from the harmfulinfluence of impulsive noise, it is highly probable that the data levelof the subject pixel Pn is within a range between the maximal value MAXoand the minimal value MINo. On the contrary, if any significant noise issuperimposed on the subject pixel Pn, there is a high probability thatthe data level thereof is out of the rang between the maximal value MAXoand the minimal value MINo.

In case the data level of the subject pixel Pn is within the rangebetween the maximal value MAXo and the minimal value MINo, harmfulinfluence of the noise is considered to be negligible and therefore thesubject pixel data Dpn is outputted as it is. However, if the subjectpixel data Dpn is outside the range between maximal value MAXo andminimal value MINo, it is assumed that significant noise is superimposedthereon, so that the subject pixel data Dpn needs to be replaced withother data.

In this case, considering the gaussian distribution characteristics ofthe noise, it is appropriate that when the subject pixel data Dpnexceeds the maximal value MAXo, the data Dpn is replaced with themaximal value MAXo; and when the subject pixel data Dpn is less than theminimal value MINo, the data Dpn is replaced with the minimal valueMINo.

The comparison signals Sc₁ and Sc₂ outputted from comparators 309 and310 respectively are as follows:

(a) If subject pixel data Dpn>maximal value MAXo, the comparison signalsbecome Sc₁ ="1" and Sc₂ ="1" to consequently form a decision signal SJ(=01) for selection of the maximal value MAXo;

(b) If maximal value MAXo≧subject pixel data Dpn ≧minimal value MINo,the comparison signals become Sc₁ ="0" and Sc₂ ="1" to consequently forma decision signal SJ (=11) for selection of the subject pixel data Dpn;

(c) If minimal value MINo>subject pixel data Dpn, the comparison signalsbecome Sc₁ ="0" and Sc₂ ="0" to consequently form a decision signal SJ(=10) for selection of the minimal value MINo.

Accordingly, selector 304 is controlled by the above-mentioned decisionsignal SJ, and the intermediate values of the three signals areselectively produced as replacement data Dn at output terminal 313.

Thus, the subject pixel Pn is replaced with data having an intermediatevalue equal to the maximal value MAXo, the subject pixel data Dpn, orthe minimal value MINo. Even if significant noise is superimposed on thesubject pixel Pn and the sampling density is low, the subject pixel Pncan be replaced with suitable data obtained by respectively adding theoffset Δ1 to the maximal value MAX, and subtracting the offset Δ2 fromthe minimal value MIN, so that it becomes possible to remove the noiseadaptively without deteriorating resolution, hence achievinghigh-fidelity reproduction of the original image.

In the embodiment described above, the offsets Δ1 and Δ2 are added toand subtracted from the maximal value MAX and the minimal value MINrespectively by means of adder 307 and subtracter 308. However, suchadjustment may alternatively be executed by employing a combinedadder-subtracter. Furthermore, addition (and subtraction) of the offsetsmay not be needed in some cases.

Although in the above embodiment the one-dimensional filter is composedin the horizontal line direction, it may alternatively be composed inthe vertical direction by replacing sample delay with line delay. It isalso possible to constitute such filter by a horizontal and verticalcombination.

Hereinafter other embodiments of the video signal compressiveencoding/decoding method and apparatus of the present invention will bedescribed with reference to FIGS. 9 through 12.

FIG. 9 shows an alternative embodiment of the encoder of the invention,wherein pixel data and a bit map are stored in a memory 501. A randomaccess memory (RAM) may be used as the memory 501. The data istransferred between a data bus 502 and an external unit via an I/O port503. An average/prediction circuit 504 generates the average value oftwo selected peripheral reference pixel samples and also generates apredictive error between the average value and the value of a subjectpixel sample. The predictive error obtained from circuit 504 is fed to apredictive error discriminator 505, in which a decision is made as towhether the predictive error is greater or not than the threshold valuefed thereto from threshold data generator circuit 506. A bit mapgenerator circuit 507 generates a bit map which becomes "0" or "1"depending on whether the predictive error is smaller or greater than thethreshold value. The bit map thus produced is stored in memory 501 viadata bus 502.

A memory R/W control circuit 508 is provided for controlling both thewrite and read operations. The above-described memory 501, I/O port 503,average/prediction circuit 504, threshold data generator circuit 506 andbit map generator circuit 507 are controlled by memory R/W controlcircuit 508.

In the FIG. 9 embodiment, when "0" is transmitted as the bit map, thevalue of the subject pixel sample is replaced with the average value andis therefore the data of the subject pixel sample is not transmitted.Instead, when "1" is transmitted as the bit map, the data of the subjectpixel sample is transmitted. Thus, the variable density subsampling isperformed in such a manner that transmission of the subject pixel sampledata is determined under control in accordance with the numerical valueof the predictive error.

FIG. 10 shows an alternative embodiment of the decoder of the invention,wherein the pixel sample data and the bit map received from I/O port 603via data bus 602 are stored in memory 601. When the bit in the bit mapis "1", the received subject pixel sample data is used directly withoutmodification. Instead, when such bit is "0", the value of the receivedsubject pixel sample is interpolated with an average data signalobtained from average generator circuit 604. Bit map discriminator 605checks the bit map and makes a decision as to whether the bitcorresponding to the subject pixel to be processed is "1" or "0". Memory601, I/O port 633, and average generator circuit 604 are controlled bymemory R/W control circuit 606.

Next, the operation of the encoder of the invention will be describedwith reference to FIGS. 11 and 12.

As shown in the flow chart of FIG. 11, first the data of the elementarypixel samples denoted by ○ are transmitted in their entirety (step 701).Then the data of each subject pixel o is predicted from the averagevalue of the two upper and lower reference pixel samples ⊚ spaced apartby two lines vertically from each other (step 702). The predictive errorin this case is judged to be greater or smaller than the threshold value(step 703). When the predictive error is smaller than the thresholdvalue, "0" is transmitted as the bit map, so that the original data ofthe subject pixel sample is replaced with the predicted value, e.g.average value, instead of being transmitted (step 704). If thepredictive error exceeds the threshold value, "1" is transmitted as thebit map and therefore the original data of the subject pixel sample istransmitted (step 705).

Subsequently the data of each subject pixel sample denoted by □ ispredicted from the average value of two peripheral reference pixelsamples ⊚ or ○ spaced apart by two pixel samples horizontally from eachother (step 706). Then in the next step 707, the predictive error isjudged to be greater or smaller than the threshold value. When thepredictive error is smaller than the threshold value, "0" is transmittedas the bit map, so that the original data of the subject pixel sample isreplaced with the predicted value instead of being transmitted (step708). If, instead, the predictive error exceeds the threshold value, "1"is transmitted as the bit map and therefore the original data of thesubject pixel sample is transmitted (step 709).

Subsequently the data of each subject pixel sample Δ is predicted fromthe average value of two peripheral reference pixel samples ⊚ -o or □--□positioned in upper and lower lines (step 710). In the next step 711,the predictive error is judged to be greater or smaller than thethreshold value. When the predictive error is smaller, "0" istransmitted as the bit map, so that the original data of the subjectpixel sample is replaced with the predicted value instead ff beingtransmitted (step 712). If the predictive error is greater, "1" istransmitted as the bit map and therefore the original data of thesubject pixel sample is transmitted (step 713).

Then the data of each subject pixel data denoted by x is predicted fromthe average value of two peripheral reference pixels ○ -□, Δ-Δ, or o--□positioned at left and right sample points (step 714). The predictiveerror in this case is judged to be greater or smaller than the thresholdvalue in step 715. When the predictive error is smaller, "0" istransmitted as the bit map, so that the original data of the subjectpixel sample is replaced with the average value instead of beingtransmitted (step 716). If the predictive error is greater, "1" istransmitted as the bit map and therefore the original data of thesubject pixel sample is transmitted (step 717).

As described above, the vertical and horizontal compressions arealternately repeated and the interval is reduced by half in eachcompression, whereby the process is executed sequentially from rough orlow-density subsampling to fine or high-density subsampling.

The operation of the decoder provided on the receiving side will now bedescribed below with reference to the flow chart of FIG. 12. In thedecoder, the data of the elementary pixel sample is initially received(step 801). Then the bit map is checked and a decision is made as towhether the bit corresponding to a subject pixel sample o is "0" or "1"(step 802). When the bit is "0", the data of the elementary pixel sample⊚ is interpolated with the vertical average value of the data of theelementary pixel samples ⊚ (step 803). Meanwhile if the bit is "1", thereceived data of the subject pixel sample ○ is used directly (step 804).

Subsequently the bit map is checked and a decision is made as to whetherthe bit corresponding to a subject pixel sample □ is "0" or "1" (step805). When the bit is "0", the data of the subject pixel sample □ isinterpolated with the average value of the data of two horizontallyperipheral reference pixel samples combined as ⊚ -- ⊚ or o--o (step806). If the bit is "1", the received data of the subject pixel sample □is directly used (step 807).

Next the bit map is checked in step 808, and a decision is made as towhether the bit corresponding to a subject pixel sample Δ is "0" or "1".When the bit is "0", the data of the subject pixel sample Δ isinterpolated with the average value of the data of two verticallyperipheral reference pixel samples combined as ⊚ -o, or □--□ (step 809).If the bit is "1", the received data of the subject pixel sample Δ isdirectly used (step 810).

In the next step 811, the bit map is checked and a decision is made asto whether the bit corresponding to a subject pixel sample x is "0" or"1". When the bit is "0", the subject pixel data x is interpolated withthe average value of the data of two horizontally peripheral referencepixel samples combined as ⊚ -□, Δ-Δ, or o-□ (step 812). Meanwhile if thebit is "1", the received data of the subject pixel sample x is directlyused (step 813).

Comparing the second embodiment (described with reference to FIGS. 9 and10) with the first embodiment (described with reference to FIGS. 3-6),the hardware scale become larger due to the necessity of a one field (orone frame) memory in the second embodiment, and the required processingtime is rendered longer, so that the second embodiment is not bestsuited for still-image processing. However, since the value of eachpixel sample to be thinned out is replaced with the predicted value, theinvention achieves the advantage that the accumulation of the predictiveerrors can be reduced in the case of prediction using such pixel sample.

The stepwise encoding of the present invention realizes sequentialdisplay changeable from a rough image to a fine image without the needfor data rearrangement. Furthermore, due to the repeated processing oflocally convergent patterns, the invention minimizes the problem oferror propagation occurring in DPCM.

The results of exemplary simulations obtained by the above-describedembodiments of the present invention are listed below:

    ______________________________________                                                    Compression rate %                                                                          SN ratio (dB)                                       ______________________________________                                        Flesh color chart                                                                           24.7            44.0                                            Woman with headband                                                                         42.6            44.0                                            Weather forecast                                                                            37.1            45.2                                            Swiss landscape                                                                             62.1            45.7                                            Tulip         77.5            45.4                                            Robot         50.1            44.0                                            ______________________________________                                    

As will be understood from the results of the simulations listed above,a remarkably high signal-to-noise ratio of 44 to 45 (dB) is achieved toensure a satisfactory quality of the reproduced image, and thecompression rate is also enhanced.

In the embodiments mentioned hereinabove, the predition mode is notlimited to the average value alone, and any other suitable mode may beemployed as an alternative.

Besides the aforesaid examples where the pixel samples are eithertransmitted or thinned out, a similar effect is also attainable by,instead of thinning out the pixel samples, reducing the number of thebits thereof and transmitting merely the high-order bits alone.

Furthermore, the intervals of the elementary pixels may be changed inaccordance with the image.

It is a matter of course that, in the second embodiment also, removal ofnoise may be executed by the use of a nonlinear filter.

What is claimed is:
 1. A video signal compressive encoding method,comprising the steps of:(a) receiving a digital video signal representedby respective pixel samples of first predetermined bits, said pixelsamples including elementary pixel samples having present values andsubject pixel samples having present values; (b) determining respectiveelementary pixel samples at a predetermined rate; (c) predicting valuesof respective ones of the subject pixel samples; (d) detectingrespective predictive errors of the predicted values from the presentvalues of the pixel samples; (e) determining if each predictive error isgreater than a threshold value; (f) generating a flag each time one ofthe predictive errors is determined to not exceed the threshold value;and (g) transmitting an encoded video signal including the elementarypixel samples, the present values of the subject pixel samples for whichthe respective predictive errors are greater than the threshold value,and a compressed data signal for each subject pixel sample for which therespective predictive error is less than or equal to the thresholdvalue, each of said compressed data signals including at least one ofthe flags.
 2. The method of claim 1, also including the steps of:(h)receiving the transmitted encoded video signal; (i) predicting a subjectpixel sample value for each of said compressed data signals; (j)interpolating the respective subject pixels corresponding to therespective compressed data signals, with the respective subject pixelsample values predicted in step (i); and (k) outputting a decoded videosignal including the transmitted elementary pixel samples, therespective subject pixel samples whose present values were included inthe transmitted encoded video signal, and the respective subject pixelsamples interpolated in step (j).
 3. A video signal compressive encodingapparatus, comprising:(a) a means for receiving a digital video signalpresented by respective pixel samples of first predetermined bits, saidpixel samples including elementary pixel samples having present valuesand subject pixel samples having present values; (b) a means fordetermining respective elementary pixel samples at a predetermined rate;(c) a means for predicting values of respective ones of the subjectpixel samples; (d) a means for detecting respective predictive errors ofthe predicted values from the present values of the pixel samples; (e) ameans for determining if each predictive error is greater than athreshold value; (f) a means for generating a flag each time one of thepredictive errors is determined not exceed the threshold value; and (g)a means for transmitting an encoded video signal including theelementary pixel samples, the present values of the subject pixelsamples for which the respective predictive errors are greater than thethreshold value, and a compressed data signal for each subject pixelsample for which the respective predictive error is less than or equalto the threshold value, each of said compressed data signals includingat least one of the flags.
 4. The apparatus of claim 3, alsocomprising:(h) a means for receiving the transmitted encoded videosignal; (i) a means for predicting a subject pixel sample value for eachof said compressed data signals; (j) a means for interpolating therespective subject pixels corresponding to the respective compresseddata signals, with the respective subject pixel sample values predictedin element (i); and (k) a means for outputting a decoded video signalincluding the transmitted elementary pixel samples, the respectivesubject pixel samples whose present values were included in thetransmitted encoded video signal, and the respective subject pixelsamples interpolated in element (j).